Devices and methods for interacting with a control system that is connected to a network

ABSTRACT

A building automation system may be controlled in response to a natural language voice message. The natural language voice message may be recorded and then sent to a voice command manager via a network. The natural language voice message is then translated into a command recognizable by a building automation controller of the building automation system. Voice recognition software may be used to create a natural language text based message from the recorded natural voice message, and the natural language text based message may then be translated into the command recognizable by the building automation controller. In response to the command, the building automation controller may perform the desired action.

This is a continuation of co-pending U.S. patent application Ser. No.13/911,638, entitled DEVICES AND METHODS FOR INTERACTING WITH A CONTROLSYSTEM THAT IS CONNECTED TO A NETWORK, filed on Jun. 6, 2013, which is acontinuation-in-part of U.S. patent application Ser. No. 13/875,213entitled DEVICES AND METHODS FOR INTERACTING WITH AN HVAC CONTROLLER,filed on the May 1, 2013, now U.S. Pat. No. 10,088,853, issued Oct. 2,2018, and incorporated herein by reference in its entirety for allpurposes.

TECHNICAL FIELD

The present disclosure relates generally to control systems, and moreparticularly, to control systems that accommodate and/or facilitatecontrol of building automation and/or other devices.

BACKGROUND

Buildings, such as commercial and residential buildings, often includevarious control systems for controlling various devices. For example,buildings often include a Heating, Ventilation, and/or Air Conditioning(HVAC) system for monitoring and/or controlling various HVAC devices, asecurity system for monitoring and/or controlling various securitysystem devices, appliance controllers for controlling variousappliances, as well as other control systems and devices. What would bedesirable is an improved system for interacting with such devices.

SUMMARY

The present disclosure relates generally to control systems, and moreparticularly, to control systems that accommodate and/or facilitatecontrol of building automation and/or other devices. In one illustrativeembodiment, a network connected building automation system may becontrolled with a voice command manager, a first voice sensor device,and a second voice sensor device connected to a network. The voicecommand manager may translate natural language commands received overthe network from the first voice sensor and/or the second voice senorinto one or more commands understandable by a building automationcontroller of the building automation system. Once the command isformulated, the voice command manager may send the formulated command tothe building automation controller. In some cases, the buildingautomation controller may be an HVAC controller, a security systemcontroller, an appliance (refrigerator, washing machine, dryer, dishwasher, television, etc.) controller, or any other suitable buildingautomatic controller, as desired.

In some instances, a voice command manager of, or for, a buildingautomation system may include an input port, an output port, a memory,and a processor. In some instances, the input port may be capable ofreceiving a natural language voice command over a network to which thevoice command manager may be connected. The memory may store thereceived natural language command and the processor may be configured totranslate the natural language voice command stored in the memory intoone or more commands understandably by a building automation controller.The output port may be used to send the one or more commands to thebuilding automation controller for execution.

In an illustrative method, a building automation system may function byreceiving a natural language voice message at one or more of a firstvoice reception unit and a second voice reception unit. A naturallanguage command based on the received natural language voice messagemay be communicated to a computing device, where the natural languagecommand may be translated into a command understandable by one or morebuilding automation controllers. Once the computing device has produceda command, the command may be sent to the one or more buildingautomation controllers for execution.

The preceding summary is provided to facilitate an understanding of someof the innovative features unique to the present disclosure and is notintended to be a full description. A full appreciation of the disclosurecan be gained by taking the entire specification, claims, drawings, andabstract as a whole.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure may be more completely understood in consideration of thefollowing description of various illustrative embodiments in connectionwith the accompanying drawings, in which:

FIG. 1 is a schematic view of an illustrative HVAC system servicing abuilding or structure;

FIG. 2 is a schematic block diagram of an illustrative network connectedbuilding automation system;

FIG. 3 is a schematic block diagram of an illustrative buildingautomation system connected by a first network and a second network;

FIG. 4 is a schematic block diagram of an illustrative network connectedbuilding automation system with a plurality of illustrative voicereception units;

FIG. 5 is a schematic block diagram of an illustrative network connectedbuilding automation system with a plurality of illustrative buildingautomation devices;

FIG. 6 is a schematic block diagram of an illustrative network connectedbuilding automation system with a plurality of illustrative buildingautomation devices and a plurality of illustrative voice receptionunits;

FIG. 7 is a schematic block diagram of an illustrative network connectedbuilding automation system with an illustrative translating mechanismand illustrative voice reception software located in illustrative voicereception units;

FIG. 8 is a schematic block diagram of an illustrative network connectedbuilding automation system with an illustrative voice command manager;and

FIG. 9 is a schematic flow diagram of an illustrative method ofoperating a building automation system.

While the disclosure is amenable to various modifications andalternative forms, specifics thereof have been shown by way of examplein the drawings and will be described in detail. It should beunderstood, however, that the intention is not to limit aspects of thedisclosure to the particular illustrative embodiments described. On thecontrary, the intention is to cover all modifications, equivalents, andalternatives falling within the spirit and scope of the disclosure.

DESCRIPTION

The following description should be read with reference to the drawingswherein like reference numerals indicate like elements throughout theseveral views. The description and drawings show several embodimentswhich are meant to illustrative in nature.

FIG. 1 is a schematic view of a building 2 having an illustrativebuilding automation system 4. While FIG. 1 shows a typical forced airtype HVAC system as an illustrative building automation sub-system 12 ofa building automation system 4, other building automation sub-systems 12of a building automation system 4 are contemplated including, but notlimited to, security systems, lighting control systems, water heatersystems (e.g., boiler systems), refrigerators, clothes washers, clothesdryer, ovens, garage doors, radiant heating systems, electric heatingsystems, cooling systems, heat pump systems, any other suitablesub-system 12 of building automations systems 4, and/or portionsthereof, as desired. The illustrative HVAC system of FIG. 1 includes oneor more HVAC components 6, a system of ductwork and air vents includinga supply air duct 10 and a return air duct 14, and one or more buildingautomation controllers 18 (e.g., HVAC controllers). The one or more HVACcomponents 6 may include, but are not limited to, a furnace, a heatpump, an electric heat pump, a geothermal heat pump, an electric heatingunit, an air conditioning unit, a humidifier, a dehumidifier, an airexchanger, an air cleaner, a damper, a valve, and/or the like.

It is contemplated that a building automation controller(s) 18 may beconfigured to activate and deactivate the building automationsub-system(s) 12 (e.g., the HVAC system) or components of the buildingautomation controller(s) or sub-system(s) (e.g., HVAC components 6) ofthe building automation system 4 in a controlled manner (e.g., tocontrol the comfort level in the structure or building 2 and/orotherwise operate electronic features of the building 2). The buildingautomation controller(s) 18 may be configured to control the buildingautomation controller devices or building automation sub-systems 12 viaa wired or wireless communication link 20. In some cases, the buildingautomation controller(s) 18 may be a thermostat, such as, for example, awall mountable thermostat or intelligent power switch (e.g., forcontrolling appliances not equipped with communications capabilities andother appliances), but this is not required in all instances. An examplethermostat may include (e.g. within the thermostat housing) or haveaccess to a temperature sensor for sensing an ambient temperature at ornear the thermostat. In some instances, the building automationcontroller(s) 18 may be a zone controller, or may include multiple zonecontrollers each monitoring and/or controlling the comfort level withina particular zone in the building or other structure.

In the illustrative building automation system 4 shown in FIG. 1, theHVAC component(s) 6 (e.g., components of the building automationcontroller or building automation sub-system 12) may provide heated air(and/or cooled air) via the ductwork 10, 14 throughout the building 2.As illustrated, the HVAC component(s) 6 may be in fluid communicationwith every room and/or zone in the building 2 via the ductwork 10 and14, but this is not required. In operation, when a heat call or commandsignal is provided by the building automation controller(s) 18, an HVACcomponent 6 (e.g. forced warm air furnace) may be activated to supplyheated air to one or more rooms and/or zones within the building 2 viasupply air ducts 10. The heated air may be forced through the supply airduct 10 by a blower or fan 22. In this example, the cooler air from eachzone may be returned to the HVAC component 6 for heating via return airducts 14. Similarly, when a cool call or command signal is provided bythe building automation controller(s) 18, an HVAC component 6 (e.g. airconditioning unit) may be activated to supply cooled air to one or morerooms and/or zones within the building or other structure via supply airducts 10. The cooled air may be forced through the supply air duct 10 bythe blower or fan 22. In this example, the warmer air from each zone maybe returned to the HVAC component 6 (e.g. air conditioning unit) forcooling via return air ducts 14. In some cases, the building automationsystem 4 may include an internet gateway 20 (e.g., a modem or otherdevice providing a communication link) or other device that may allowone or more of the HVAC components 6, as described herein, tocommunicate over a wide area network (WAN) such as, for example, theInternet, and/or a device 20 that may allow one or more HVAC components6 to communicate over any other network.

In some cases, the system of vents or ductwork 10 and/or 14 can includeone or more dampers 24 to regulate the flow of air, but this is notrequired. For example, one or more dampers 24 may be coupled to one ormore HVAC controller(s) 18, and can be coordinated with the operation ofone or more HVAC components 6. The one or more HVAC controller(s) 18 mayactuate dampers 24 to an open position, a closed position, and/or apartially opened position to modulate the flow of air from the one ormore HVAC components 6 to an appropriate room and/or zone in thebuilding or other structure. The dampers 24 may be particularly usefulin zoned HVAC systems, and may be used to control which zone(s) receivesconditioned air from the HVAC component(s) 6.

In many instances, one or more air filters 30 may be used to remove dustand other pollutants from the air inside the building 2. In theillustrative example shown in FIG. 1, the air filter(s) 30 is installedin the return air duct 14, and may filter the air prior to the airentering the HVAC component 6, but it is contemplated that any othersuitable location for the air filter(s) 30 may be used. The presence ofthe air filter(s) 30 may not only improve the indoor air quality, butmay also protect the HVAC components 6 from dust and other particulatematter that would otherwise be permitted to enter the HVAC component 6.

In some cases, and as shown in FIG. 1, the illustrative buildingautomation system 4 may include an equipment interface module (EIM) 34.When provided, the equipment interface module 34 may be configured tomeasure or detect a change in a given parameter between the return airside and the discharge air side of the building automation system 4. Forexample, the equipment interface module 34 may be adapted to measure adifference in temperature, flow rate, pressure, or a combination of anyone of these parameters between the return air side and the dischargeair side of the building automation controller or building automationsub-system 12. In some cases, the equipment interface module 34 may beadapted to measure the difference or change in temperature (delta T)between a return air side and a discharge air side of the HVAC systemfor the heating and/or cooling mode. The delta T for the heating modemay be calculated by subtracting the return air temperature from thedischarge air temperature (e.g. delta T=discharge air temp.-return airtemp.). For the cooling mode, the delta T may be calculated bysubtracting the discharge air temperature from the return airtemperature (e.g. delta T=return air temp.-discharge air temp.).

In some cases, the equipment interface module 34 may include a firsttemperature sensor 38a located in the return (incoming) air duct 14, anda second temperature sensor 38b located in the discharge (outgoing orsupply) air duct 10. Alternatively, or in addition, the equipmentinterface module 34 may include a differential pressure sensor includinga first pressure tap 39a located in the return (incoming) air duct 14,and a second pressure tap 39b located downstream of the air filter 30 tomeasure a change in a parameter related to the amount of flowrestriction through the air filter 30. In some cases, the equipmentinterface module 34, when provided, may include at least one flow sensorthat is capable of providing a measure that is related to the amount ofair flow restriction through the air filter 30. In some cases, theequipment interface module 34 may include an air filter monitor. Theseare just some examples.

When provided, the equipment interface module 34 may be capable ofand/or configured to communicate with the building automationcontroller(s) 18 via, for example, a wired or wireless communicationlink 42. In other cases, the equipment interface module 34 may beincorporated or combined with the building automation controller 18. Ineither case, the equipment interface module 34 may communicate, relay orotherwise transmit data regarding the selected parameter (e.g.temperature, pressure, flow rate, etc.) to the building automationcontroller 18. In some cases, the building automation controller 18 mayuse the data from the equipment interface module 34 to evaluate thesystem's operation and/or performance. For example, the buildingautomation controller 18 may compare data related to the difference intemperature (delta T) between the return air side and the discharge airside of the building automation system 4 to a previously determineddelta T limit stored in the building automation controller 18 todetermine a current operating performance of the building automationsystem 4. Depending upon the application and/or where the buildingautomation system user is located, remote access and/or control of thebuilding automation controller 18 may be provided over a first network54 a (e.g. a local WiFi network) and/or a second network 54 b (e.g. theInternet), as shown in FIG. 3. In some instances, the first network 54 aand the second network 54 b may collectively be considered network 54.An example of network connected building automation system is describedin U.S. patent application Ser. No. 13/559,470, entitled METHOD OFASSOCIATING AN HVAC CONTROLLER WITH AN EXTERNAL WEB SERVICE, filed Jul.26, 2012, and hereby incorporated by reference in its entirety. Anotherexample of a network connected building automation system is describedin U.S. patent application Ser. No. 13/875,213, entitled DEVICES ANDMETHODS FOR INTERFACING WITH AN HVAC CONTROLLER, filed May 1, 2013, andhereby incorporated by reference in its entirety.

In some instances, a variety of remote devices 62 (see FIG. 4) may beused to access and/or control the building automation controller 18 froma remote location (e.g. remote from building automation controller 18and interior or exterior the building 2) over a network 54 (e.g. thefirst network 54 a and/or second network 54 b). The remote devices 62may include, but are not limited to, mobile phones including smartphones, personal digital assistants (PDAs), tablet computers, laptop orpersonal computers, wireless network-enabled key fobs, e-Readers and/orother computing devices, as desired. In some instances, the remotedevice 62 may include a voice sensor device or voice reception unit 16,as shown in FIG. 4.

In many cases, the remote devices 62 may be a remote wireless devices 62that is configured to communicate wirelessly over the first network 54 aand/or second network 54 b with the building automation controller 18via one or more wireless communication protocols such as cellularcommunication, ZigBee, REDLINK™, Bluetooth, WiFi, IrDA, dedicated shortrange communication (DSRC), EnOcean, and/or any other suitable common orproprietary wireless protocol, as desired.

In some cases, and as shown in FIG. 4, a building automation system 4serving a building 2 may include a network 54 (e.g., a local areanetwork and/or wide area network), a voice command manager 8 connectedto the network 54, one or more building automation controllers 18, oneor more voice sensor devices or voice reception units 16 separate fromand/or included in the building automation controller(s) 18, and/or oneor more building automation or sub-system(s) 12. The voice commandmanager 8 may be implemented on and/or include a server 26 having amemory 28 and a processor 32 and/or other computing device having amemory and processor. In some instances, the server 26 may include oneor more servers and/or computing devices.

The server 26 or other computing device on which the voice commandmanager 8 is implemented may be located at any location with respect tothe building automation controllers or sub-systems 12 and/or the voicesensor devices or voice reception units 16, as desired. In someinstances, the voice command manager 8 may be implemented on a server 26located outside of the building 2, or a server 26 located inside of thebuilding 2. Alternatively, or in addition, one or more portions of thevoice command manager 8 may be stored on a server 26 located inside ofthe building 2 and one or more portions of the voice command manager 8may be stored on a server located outside of the building 2.

The voice command manager 8 may take on any form of a server 26 and/orthe voice command manager may include the server 26. Illustratively, theserver 26 may be or may be included in a computing device or specializedvoice recognition computer (e.g., a computer comprising hardware capableof storing voice recognition software and/or natural language commandtranslation software), where the server 26 may include software capableof responding to requests across the network 54 to provide a service.Where the voice command manager 8 is on a single server 26 includingcomputer or computing devices that communicate with the buildingautomation controllers or sub-systems 12, users of the buildingautomation system 4 may only need to have the voice command manager 8recognize their individual voice commands, as opposed to trainingseveral voice command managers at each building automation controller orsub-system 12 (e.g., a voice command manager at each of a clotheswasher, a clothes dryer, a refrigerator, a thermostat, a securitysystem, a television, etc.) separately. Additionally, a single voicecommand manager 8 may allow for training the software of the voicecommand manager 8 (e.g., training the software to become accustomed to abuilding automation system user's commands) from a single location(e.g., from a comfortable chair) instead of at a building automationcontroller or sub-system 12 (e.g., a clothes washer, a clothes dryer, arefrigerator, a thermostat, a security system, a television, etc.)having an individual voice command manager 8.

In some instances, and as best shown in FIG. 2, the voice commandmanager 8 may include an input port 36, a memory 28, a processor 32, andan output port 40. The memory 28 may be any suitable type of storagedevice including, but not limited to, RAM, ROM, EPROM, flash memory, ahard drive, and/or the like. In some cases, the processor 32 may storeinformation within the memory 28, and may subsequently retrieve thestored information from the memory 28.

Illustratively, the input port 36 may receive a natural language voicecommand or message (e.g., an audible verbal command or message) over anetwork 54 (e.g., a home network, WiFi network, a network connected tothe world wide web or other wide area network, a cellular network, atelephone network, or any other wired or wireless communication network)from one or more voice sensor devices or voice reception units 16 orother devices capable of transmitting voice patterns to the voicecommand manager 8. In some instances, the natural language voice commandor message may be communicated in a compressed or uncompressed audiofile format. Formats may include, for example, WAV, AIFF, AU or rawheader-less PCM, FLAC, Monkey's Audio (filename extension APE), WavPack(filename extension WV), TTA, ATRAC Advanced Lossless, Apple Lossless(filename extension m4a), MPEG-4 SLS, MPEG-4 ALS, MPEG-4 DST, WindowsMedia Audio Lossless (WMA Lossless), Shorten (SHN), MP3, Vorbis,Musepack, AAC, ATRAC and Windows Media Audio Lossy (WMA lossy), or anyother suitable format, as desired.

The memory 28 may receive the natural language voice command(s) ormessage(s) either directly from the input port 36 or the memory 28 mayreceive the natural language voice command(s) or message(s) from theprocessor 32. The processor 32 may receive the natural languagecommand(s) or message(s) from the input port 36 or the memory 28 and maytranslate the natural language command(s) or message(s) into one or morecommands understandable by the building automation controller(s) orsub-system(s) 12 (e.g., machine language binary, hexadecimal code,etc.). The output port 40 may then send the one or more commandsunderstandable by the building automation controllers or sub-system(s)12 to the building automation controller or sub-system(s) 12 forexecution.

In some instances, the input port 36 of the voice command manager 8 maybe in communication via a network 54 (e.g., a wireless or wirednetwork). The network 54 may be confined within the building 2 or mayextend inside and/or outside of the building. While in communicationwith the network 54, the input port 36 may receive the natural languagevoice command(s) or message(s) from one or more voice sensor device(s)or voice reception unit(s) 16 that may be or has been in communicationwith the network 54. In some instances, the voice sensor device(s) orvoice reception unit(s) 16 may be within the building 2 and may or maynot be connected to the network 54. Alternatively, or in addition, thevoice sensor device(s) or voice reception unit(s) 16 may be locatedremotely with respect to the building 2 and may communicate with theinput port 36 and/or the output port 40 of the voice command manager 8via any network connected to the world wide web (e.g., the internet) toprovide a commanded to one or more building automation controllers orsub-systems 12 over the network 54.

In some illustrative examples, and as best shown in FIG. 4, the buildingautomation system 4 may include a first voice sensor device or voicereception unit 16 a (e.g., a voice sensor device of voice reception unit16 of a remote device 62) and a second voice sensor device or voicereception unit 16 b (e.g., a voice reception unit 16 of a buildingautomation controller 18). The first and/or second voice sensor devicesor voice reception units 16 a, 16 b may be in communication with acommon voice command manager 8 over the network 54. In some instances,the first and/or second voice sensor devices or voice reception units 16a, 16 b may receive and/or store natural language commands from a userand communicate the natural language commands to the voice commandmanager 8 over the network 54. Additionally, in some instances, thefirst and/or second voice sensor devices or voice reception units 16 a,16 b may receive, display and/or play acknowledgements from the voicecommand manager 8 and/or the building automation controllers orsub-systems 12. For example, the voice command manager may indicate thatanother user is currently speaking to the system. In another example,when a first user provides a voice command to the first voice sensordevice or voice reception unit 16 a and a second user provides a voicecommand to the second voice sensor device or voice reception unit 16 bthat overlaps in time with the first user's command, the voice commandmanager 8 may acknowledge both voice commands and communicate viaaudible verbal communication (and/or a visual message on a screen) whichcommand was accepted and why (e.g., “Terry just made a change to thesystem via a smart phone”). In yet another example, the voice commandmanager may acknowledge to the user that a command was received and thatan action was taken, such as “the temperatures has been adjusted toseventy-two degrees”.

Using the memory 28 and/or the processor 32, the voice command manager 8may translate the natural language commands received from the firstvoice sensor device or first voice reception unit 16 a and/or the secondvoice sensor device or voice reception unit 16 b into one or morecommands understandable by the building automation controller orsub-system 12. Once the natural language commands have been translatedinto one or more commands understandable by the building automationcontroller or sub-system 12, the voice command manager 8 may communicateor transmit the one or more commands to the building automationcontroller or sub-system 12 for execution.

In instances where the first and second voice sensor devices or voicereception units 16 a, 16 b communicate a received natural languagecommand for the building automation controller or sub-system 12 to thevoice command manager 8, the voice command manager 8 may translate bothof the received natural language commands into corresponding commandsunderstandable by the building automation controller or sub-system 12.If there is an identified conflict between the translated commandsunderstandable by the building automation controller or sub-system 12,the voice command manager 8 may determine which of the correspondingtranslated commands, if any, have priority is/are communicated to thebuilding automation controller or sub-system 12.

Alternatively, the voice command manager 8 may determine and/or identifyif there is a conflict in the received natural language commands for thebuilding automation controller or sub-system 12 prior to translating thenatural language commands to commands understandable by the buildingautomation controller or sub-system 12. Then, if there is an identifiedconflict between the received natural language commands, the voicecommand manager 8 may determine which of the natural language commands,if any, should be translated and communicated to the building automationcontroller or sub-system 12.

The voice command manager 8 or other feature (e.g., building automationcontroller or sub-system 12, etc.) may be capable of communicating witha user through one or more of the voice sensor device(s) or voicereception unit(s) 16. In one example, voice command manager 8 maycommunicate back to the first or second voice sensor devices or voicereception units 16 a, 16 b, or both, in response to receiving one ormore natural language commands for a building automation controller(s)or sub-system(s) 12 from one or more of the first and second voicesensor devices or voice reception units 16 a, 16 b. In some instances,if two or more natural language commands for the building automationcontroller(s) or sub-system(s) 12 are received, the voice commandmanager 8 may communicate which command(s), if any, were translated andcommunicated to the building automation controller(s) or sub-systems 12.In one example, if a first user at first time earlier than a second timetells the thermostat to set back to a temperature at 2:00 PM and asecond user (or the first user) at the second time tells a thermostat toset back at 3:00 PM for vacation, the voice command manager 8 mayidentify the conflict, report to the second user that there is aconflict in the thermostat scheduling, and ask the second user if thesecond user would like to override the command by the first user to setback the temperature at 2:00 PM. The communication (e.g., anacknowledgement) from the voice command manager 8 to the voice sensordevice or voice reception unit(s) 16 may include a natural languageacknowledgements, audible verbal message, a text-based message, a lightindicator, an audible indicator, or any other communication, as desired.

In some instances, the voice sensor device or voice reception unit(s) 16may perform other functions. For example, the voice sensor device orvoice reception unit 16 may be capable of handling security and/orspeaker identification instead of, or in addition to, programming eachbuilding automation controller or sub-system 12 separately to handlesecurity and/or speaker identification, which may have various differentsecurity schemes to protect the devices or sub-systems 12 fromunintended users operating them or from other undesirable occurrences.

It is contemplated that the voice sensor device(s) or voice receptionunit(s) 16 may be any type of device. For example, the voice sensordevice(s) or voice reception unit(s) 16 may be any device capable ofreceiving an audible verbal communication (e.g., a natural languagecommand), storing (at least temporarily) the received audible verbalcommunication, and communicate the received audible verbal communicationto a computing device (e.g., the voice command manager 8 or a computingdevice maintaining the voice command manager 8), receiving verbal ortext-based communications from the computing device or other device ofthe building automation system 4, and/or displaying verbal or text-basedcommunications from the computing device or other device of the buildingautomation system 4 or displaying an indicator of the receivedcommunications from the computing device or other device of the buildingautomation system 4. Illustratively, the voice sensor device(s) or voicereception unit(s) 16 may be or may be included in a telephone (e.g., asmart phone or other telephone), a thermostat, a security system panel,a sensor module, a tablet computer, a desktop computer, a lap topcomputer, a dedicated voice reception panel, any other type of controlpanel, and/or any other capable device, or any combination thereof. Thevoice sensor device(s) or voice reception unit(s) 16 may be permanentlylocated within the building 2, may be permanently located remotely withrespect to the building 2, and/or may be positionally adjustable withrespect to being interior and/or exterior of the building 2.

In some instances, the building automation system 4 may include two ormore building automation controllers or sub-systems 12 (e.g., two ormore of HVAC systems, security systems, lighting control systems, waterheater systems, refrigerators, clothes washers, clothes dryers, ovens,garage door opening systems, appliances, etc). In a building automationsystem 4 with a first building automation controller or sub-system 12 a(e.g., a first appliance) and a second building automation controller orsub-system 12 b (e.g., a second appliance), as shown in FIGS. 3, 5 and6, a voice sensor device or voice reception unit 16 may receive anatural language message or phrase for one or more of the first andsecond building automation controllers or sub-systems 12 a, 12 b. Thevoice sensor device or voice reception unit 16 may then pass the naturallanguage message or phrase (in either audible form or text form) to thevoice command manager 8 including and/or implemented on a computingdevice (e.g., a server 26 with a processor 32 and memory 28) where thenatural language message or phrase may be translated into a commandunderstandable by one or more of the first and second buildingautomation controllers or sub-systems 12 a, 12 b and is sent to the oneor more of the first and second building automation controllers orsub-systems 12 a, 12 b.

In some instances, a translating mechanism 44 of the voice commandmanager 8, as shown in FIGS. 7 and 8, may receive the natural languagemessage or phrase and translate the natural language message or phraseinto a command understandable by one or more of the first and secondbuilding automation controller or sub-systems 12 a, 12 b. Providing acentral voice translating mechanism 44 (and voice command manager) mayhelp the inputs and outputs of the building automation system 4 to beconsistent regardless of the type (e.g., brand or version) of a buildingautomation interface (e.g., the remote devices 62, the buildingautomation controllers 18, and/or the building automation controllers orsub-systems 12). Additionally, the translating mechanism 44 or any otherprocessing feature of the voice command manager 8 may help determinewhich of the first and/or second building automation controller orsub-system 12 a, 12 b the translated command is to be sent.Illustratively, the translating mechanism 44 may translate the naturallanguage message or phrase to a command understandable by one or more ofthe first and second building automation controller or sub-system 12 a,12 b before, during, or after making a determination to which, if any,of the first and second building automation controllers or sub-systems12 a, 12 b to send the command.

The translating mechanism 44 may be configured to operate by executinginstructions 46 (e.g., with the processor 32) stored on non-transitorycomputer readable media (e.g., memory 28). The instructions 46 stored onthe non-transitory computer readable media may allow the translatingmechanism 44 to translate a natural language text or voice based messageor phrase into a command understandable by building automationcontrollers or sub-systems 12. Additionally, the stored instructions 46may facilitate allowing the translating mechanism 44 to make decisions.For example, the stored instructions may allow the translating mechanism44 to make decision as to which command(s) from incoming naturallanguage messages or phrases to pass along to the building automationcontrollers or sub-systems 12, which command(s) to send to whichbuilding automation controllers or sub-systems 12, whichacknowledgements of actions taken or commands sent to send to which ofthe voice sensor device(s) or voice reception unit(s) 16, and/or anyother decisions, as desired.

In some cases, there may be a plurality of translating mechanisms 44 inone or more voice command managers 8. In other cases, there may be asingle translating mechanism 44 in a single voice command manager 8. Ininstances, where there is a single translating mechanism 44 or othertranslating feature of the voice command manager 8, less computing powermay be utilized than when there are multiple translating mechanisms 44or translating features. A translating mechanism 44 may operate bylearning and attempting to better its accuracy in forming buildingautomation controller or sub-system 12 commands from various naturallanguage messages, phrases, or commands. As a result, when there is asingle translating mechanism 44, only a single translating mechanism ofthe voice command manager 8 may need to learn how to interpret naturallanguage messages, phrases, or commands into commands understandable bythe building automation controller(s) or sub-system(s) 12.

In some instances, voice recognition software 48 may be utilized toconvert a natural language message, phrase, or command into a text-basednatural language message, phrase, or command that may then be translatedwith the translating mechanism 44 into the appropriate commandunderstandable by the building automation controller or sub-system 12.Illustratively, the voice recognition software 48 may include a libraryof voice commands that interface with the server 26 (e.g., thetranslating mechanism 44 of the voice command manager 8) to providecommands to the building automation controllers or sub-systems 12. Thevoice recognition software 48 may be any type of software used torecognize voice messages, phrases, and/or commands, and interpret theminto text based voice messages, phrases, and/or commands. In oneexample, the voice recognition software 48 may be DRAGON NATURALLYSPEAKING™ voice recognition software. Additionally, or alternatively,other similar or different voice recognition software 48 may beutilized, such as the software executing the Siri voice recognitionfeature of the Apple™ Iphone™.

The voice recognition software 48 may be stored at the computing device(e.g., in memory 28) of the voice command manager 8, as shown in FIG. 7.Alternatively, or in addition, the voice recognition software 48 may bestored at the voice sensor devices or voice reception units 16, as shownin FIG. 8. Illustratively, the voice recognition software 48 may bestored at one or more locations of the building automation system 4(e.g., at multiple voice sensor devices or voice reception units 16, atthe voice command manager 8 and one or more voice sensor devices orvoice reception units 16, or at any other desired location).

An illustrative, but non-limiting method of a building automation system4 used to control and/or monitor building automation controllers orsub-systems in one or more buildings 2 is shown in FIG. 9. At block 102,the building automation system 4 may receive a natural language voicecommand or message at one or more voice sensor devices or voicereception units 16 (e.g., a first voice sensor device or voice receptionunit 16 a and a second voice sensor device or voice reception unit 16b). In some instances, the natural language command may be the naturallanguage voice message. Alternatively, or in addition, the naturallanguage command may be a text string derived from the natural languagevoice message via voice recognition software 48. At block 104, thebuilding automation system 4 may communicate the natural languagecommand or message to a computing device (e.g., the server 26 or othercomputing device). At block 106, the computing device (e.g., with thevoice command manager 8 on the server 26) may translate the communicatednatural language command or message to a command understandable by theone of the one or more building automation controllers or sub-systems12. After the command has been translated, and at block 108, thebuilding automation system 4 may send the command understandable by theone or more building automation controllers or sub-systems 12 to the oneor more building automation controllers or sub-systems 12 for executionby the one or more building automation controllers or sub-systems 12. Insome cases, the building automation system 4 may determine whichsub-system(s) 12 the command pertains, and only send the command to theappropriate sub-system(s) 12.

In some instances, the building automation system 4 may receive aplurality of natural language voice messages from a plurality of voicesensor devices or voice reception units 16 in proximity in time. Theplurality of natural language voice messages may include two or morecommands for a single building automation controller or sub-system 12 orfor a plurality of building automation controllers or sub-system 12. Ininstances where the plurality of natural language voice messages includetwo or more commands for a single building automation controller orsub-system 12, the computing device may select which, if any, of the twoor more commands for the single building automation controller orsub-system 12 to send to the single building automation controller orsub-system 12 as a command understandable by the building the singlebuilding automation controller for execution thereby. The computingdevice may execute a number of rules that establish a priority scheme tohelp make this selection.

In some instances, the building automation system 4 may provide acommunication to a user that an action was taken in response to acommand to the building automation controllers or sub-systems 12. In oneexample, through one or more of the first voice sensor device or voicereception unit 16 a and the second voice sensor device or voicereception unit 16 b, a verbal communication of an action taken or nottaken by the building automation controller(s) or sub-system(s) 12 maybe provided in response to the command received by that buildingautomation controller or sub-system 12. Where a plurality of commandwere received by one or more building automation controller(s) orsub-system(s) 12 in close proximity in time to one another, a pluralityof acknowledgements may be made through the voice sensor device(s) orvoice reception units 16 from which the natural language voice messageswere received. Although method 100 may be described as being performedin a particular order, it is contemplated the method may be performed inone or more other orders, as desired.

Having thus described several illustrative embodiments of the presentdisclosure, those of skill in the art will readily appreciate that yetother embodiments may be made and used within the scope of the claimshereto attached. Numerous advantages of the disclosure covered by thisdocument have been set forth in the foregoing description. It will beunderstood, however, that this disclosure is, in many respect, onlyillustrative. Changes may be made in details, particularly in matters ofshape, size, and arrangement of parts without exceeding the scope of thedisclosure. The disclosure's scope is, of course, defined in thelanguage in which the appended claims are expressed.

What is claimed is:
 1. A system of controlling a network connectedbuilding automation system servicing a building, comprising: a network;a server located outside of the building and connected to the network; avoice command manager implemented on the server; a thermostat locatedinside of the building, the thermostat comprising a temperature sensorand a first voice sensor device, the thermostat is in communication withthe voice command manager over the network and the thermostat isconfigured to receive natural language voice commands from a user viathe first voice sensor device, and to communicate the received naturallanguage voice commands in an audio file to the voice command managerover the network without the thermostat attempting to recognize naturallanguage voice commands in the audio file; a second voice sensor devicein communication with the voice command manager over the network, thesecond voice sensor device configured to receive natural language voicecommands from a user via the second voice sensor device, and tocommunicate the natural language voice commands to the voice commandmanager over the network; and wherein the voice command managertranslates the natural language voice commands received from thethermostat and the natural language voice commands received from thesecond voice sensor device into one or more commands understandable bythe thermostat, and transmits the one or more commands to the thermostatfor execution.
 2. The system of claim 1, wherein when the thermostatcommunicates a natural language voice command received by the thermostatfrom a user via the first voice sensor device to the voice commandmanager, and the second voice sensor device communicates a naturallanguage voice command received by the second voice sensor device from auser to the voice command manager, the voice command manager translatesboth of the received natural language voice commands into correspondingcommands understandable by the thermostat, and if there is an identifiedconflict, determine which of the corresponding commands, if any, iscommunicated to the thermostat for execution.
 3. The system of claim 2,wherein the voice command manager communicates a user notification backto the thermostat, the second voice sensor device, or both, identifyingwhich command of the corresponding commands was communicated to thethermostat for execution.
 4. The system of claim 3, wherein the usernotification is in a natural language format.
 5. The system of claim 1,wherein the second voice sensor device is a phone.
 6. A systemcomprising: a voice command manager for a building automation system ofa building, the voice command manager comprising: an input port forreceiving two or more natural language voice commands over a network; amemory for storing the received natural language voice commands; aprocessor configured to translate each of the two or more naturallanguage voice commands into one or more translated commandsunderstandable by a building automation controller of the buildingautomation system; an output port for sending the one or more translatedcommands understandable by the building automation controller to thebuilding automation controller for execution; and wherein the processoris further configured to identify if there is a conflict between two ormore of the natural language voice commands, and if so, determine whichone of the conflicting two or more natural language voice commands is tobe: translated into one or more commands understandable by the buildingautomation controller; provided to the output port; and sent to thebuilding automation controller for execution, so that only one of theconflicting two or more natural language voice commands is executed; andthe building automation controller configured to execute the sentcommand.
 7. The system of claim 6, wherein the voice command manager isimplemented in a server outside of the building.
 8. The system of claim6, wherein the voice command manager is implemented in a computer insideof the building, and wherein the input port of the voice command manageris in communication with a wireless network and receives at least one ofthe natural language voice commands via the wireless network.
 9. Thesystem of claim 8, wherein the input port receives at least one of thenatural language voice commands from a voice sensor device locatedwithin the building via the wireless network.
 10. The system of claim 6,wherein the input port receives at least one of the natural languagevoice commands from a voice sensor device located within the building.11. The system of claim 10, wherein the voice sensor device is one ormore of a thermostat, a security system panel, a sensor module, and avoice reception device.
 12. The system of claim 6, wherein the inputport receives at least one of the natural language voice commands from avoice sensor device located outside of the building.
 13. The system ofclaim 12, wherein the voice sensor device is one or more of a smartphone, a tablet computer, a desktop computer, and a lap top computer.14. The system of claim 6, wherein the building automation systemincludes at least two sub-systems, wherein the processor is configuredto determine which of the at least two sub-systems to send each of theone or more translated commands, and then sends each of the one or moretranslated commands to the appropriate sub-system via the output portfor execution.
 15. The system of claim 14, wherein the at least twosub-systems include one or more of an HVAC system, a security system, alighting control system, a water heater system, a refrigerator, aclothes washer, a clothes dryer, and an oven.
 16. A method ofinterfacing with a building automation controller of a buildingautomation system, the method comprising: receiving a natural languagevoice message that includes a natural language command for the buildingautomation controller of the building automation system; communicatingthe natural language voice message in an audio file format to a serverthat is located remotely from and in operative communication with thebuilding automation controller without the building automationcontroller first attempting to recognize the natural language command inthe natural language voice message; translating the natural languagevoice message with the server to a translated building controllercommand understandable by the building automation controller of thebuilding automation system; sending the translated building controllercommand to the building automation controller of the building automationsystem for execution; and executing the translated building controllercommand with the building automation controller.
 17. The method of claim16, further comprising: through the building automation controller,providing a verbal communication of an action taken by the buildingautomation controller in response to the translated building controllercommand.
 18. The method of claim 16, further comprising: receiving aplurality of natural language voice messages, where the plurality ofnatural language voice messages comprise two or more conflictingcommands for the building automation controller of the buildingautomation system; and selecting with the server which of the two ormore conflicting commands to send to the building automation controllerfor execution.
 19. The method of claim 18, wherein the selecting stepincludes sending a query in a natural language format to a user to querywhich of the two or more conflicting commands to send to the buildingautomation controller for execution, and receive a correspondingresponse from the user.
 20. The method of claim 16, wherein the buildingautomation controller comprises a thermostat.